Computer Vision, User Segment, and Missing Item Determination

ABSTRACT

Techniques and systems are described that leverage computer vision as part of search to expand functionality of a computing device available to a user and increase operational computational efficiency as well as efficiency in user interaction. In a first example, user interaction with items of digital content is monitored. Computer vision techniques are used to identify digital images in the digital content, objects within the digital images, and characteristics of those objects. This information is used to assign a user to a user segment of a user population which is then used to control output of subsequent digital content to the user, e.g., recommendations, digital marketing content, and so forth.

RELATED APPLICATIONS

This application claims priority to U.S. patent application Ser. No.17/141,618, filed Jan. 5, 2021, titled “Computer Vision, User Segment,and Missing Item Determination”, which claims priority to U.S. patentapplication Ser. No. 16/235,007, filed Dec. 28, 2018, titled “ComputerVision, User Segment, and Missing Item Determination”, which claimspriority under 35 U.S.C. § 119(e) to U.S. Provisional Patent ApplicationNo. 62/612,275, filed Dec. 29, 2017, and titled “Computer Vision,” theentire disclosures of which are incorporated here by reference.

BACKGROUND

Search is one of the primary techniques, via which, users of computingdevices may locate information of interest. Users, for instance, mayenter search queries to find digital content (e.g., digital images,video, music, and so on), locate products or services, and so on. Thisis typically performed as a keyword search in which a user expresses agoal of the search to the best of their ability using text. This text isthen matched to items of digital content that are tagged using similarkeywords.

Conventional search techniques, however, are typically repetitive and asa result may involve performance of repeated user interactions. A user,for instance, may initiate a search by a computing device for an item ofinterest. If that item is not found, the user is then forced to manuallyrepeat the search. This may be due to unavailability of the item ofinterest or due to a disconnect between how a goal of a search isexpressed by a user using text and how the item of interest is tagged.Forced repetition as part of these conventional systems may thus hinderoperation of the computing devices that implement the searchfunctionality and result in user frustration.

Further, search techniques may encounter additional challenges based ona configuration of a computing device, via which, the search is to beinitiated. Mobile devices, for instance, have become an integral part ofa user's everyday life. For example, a mobile phone may be used to reademails, engage in social media, capture digital images, communicate viainstant messages, and so forth. Likewise, wearable devices such as smartwatches have continued to expand this interaction. Accordingly, usershave access to a wide range of devices in a variety of usage scenarios.

However, configuration as a mobile device may introduce challenges andcomplexity in support of user interactions with these devices. A mobilephone or smart watch, for instance, may have a limited ability tosupport entry of text, navigate between files, and so on. Accordingly,user interaction with these devices may be limited and causecomputational inefficiencies as a result.

SUMMARY

Techniques and systems are described that leverage computer vision aspart of search to expand functionality of a computing device availableto a user and increase operational computational efficiency as well asefficiency in user interaction. In a first example, user interactionwith items of digital content is monitored. Computer vision techniquesare used to identify digital images in the digital content, objectswithin the digital images, and characteristics of those objects. Thisinformation is used to assign a user to a user segment of a userpopulation which is then used to control output of subsequent digitalcontent to the user, e.g., recommendations, digital marketing content,and so forth.

In a second example, techniques are described for surfacing listings ofproducts and services to client device users by attempting to determineproducts and services that these users are likely to want, and thensurfacing the determined products and services. A machine-learning model(e.g., a neural network), for instance, may be trained by the computingdevice using a plurality of training digital images. The plurality oftraining digital images depict physical environments and items disposedin the physical environments. Thus, the training digital images depict“good” examples of physical environments and the items (e.g., objects)disposed in those environments. From this, the machine-learning module“learns” which items are typically included in particular physicalenvironments and/or in combination with other items. The trained modelis then used to implement the missing item techniques. A subsequentdigital image, for instance, may be generated by a digital camera of acomputing device as a “live feed” of digital images of a physicalenvironment. A determination is then made that an item is missing in thesubsequent physical environment based on the subsequent digital imageand the trained machine-learning model by the computing device.

This Summary introduces a selection of concepts in a simplified formthat are further described below in the Detailed Description. As such,this Summary is not intended to identify essential features of theclaimed subject matter, nor is it intended to be used as an aid indetermining the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description is described with reference to the accompanyingfigures. Entities represented in the figures may be indicative of one ormore entities and thus reference may be made interchangeably to singleor plural forms of the entities in the discussion.

FIG. 1 is an illustration of an environment in an example implementationthat is operable to employ camera platform techniques described herein.

FIG. 2 depicts a system in an example implementation showing operationof a camera platform manager module of FIG. 1 in greater detail.

FIG. 3 depicts a system in an example implementation showing operationof the camera platform manager module of FIG. 1 in greater detail asemploying a user profile.

FIG. 4 depicts an example system of obtaining a recommendation based onthe generated user profile of FIG. 3 .

FIG. 5 is a flow diagram depicting a procedure in an exampleimplementation in which a user is assigned to a user segment used togenerate a recommendation based on monitored user interaction withdigital images.

FIG. 6 is a flow diagram depicting a procedure in an exampleimplementation in which monitored manipulation of a user with a digitalimage as focusing on an object or characteristic is used to assign theuser to a user segment.

FIGS. 7-10 depict user interfaces showing examples of user manipulationand focus.

FIG. 11 depicts images of an example implementation in which items thatare common to a particular context are determined from the images toenable a further determination of items that are missing from anotherimage of the particular context.

FIG. 12 depicts a scenario in an example implementation in whichaugmented and/or virtual reality (AR/VR) techniques are used to suggestitems that are missing from a particular context.

FIG. 13 depicts another scenario in an example implementation in whichAR/VR techniques are used to suggest different items that are missingfrom the particular context.

FIG. 14 is a flow diagram depicting a procedure in an exampleimplementation in which machine-learning techniques and AR digitalcontent are employed as part of missing item determination.

FIG. 15 illustrates an example system including various components of anexample device that can be implemented as any type of computing deviceas described and/or utilize with reference to FIGS. 1-14 to implementembodiments of the techniques described herein.

DETAILED DESCRIPTION

Overview

Conventional search techniques may involve repeated attempts to locate adesired product or service due to unavailability of the item ofinterest. This may also be due to a disconnect between how a goal of asearch is expressed by a user using text and how the item of interest istagged in order to locate the item. These challenges may also beexacerbated based on a type of computing device that is to initiate thesearch, e.g., due to limitations of text entry and limited display sizes(if available) for mobile devices, artificial assistant systems, and soforth. This results in computational inefficiency as well as userfrustration.

Accordingly, techniques and systems are described that leverage computervision as part of search to expand functionality of a computing deviceavailable to a user and increase operational computational efficiency aswell as efficiency in user interaction. In a first example, userinteraction with items of digital content is monitored. Computer visiontechniques are used to identify digital images in the digital content,objects within the digital images, and characteristics of those objects.This information is used to assign a user to a user segment of a userpopulation which is then used to control output of subsequent digitalcontent to the user, e.g., recommendations, digital marketing content,and so forth.

To do so, the computer vision techniques as implemented to monitormanipulation by a user via a user interface with a digital image asfocusing on an object or characteristic of the object in the digitalimage. The user, for instance, may perform a gesture which is recognizedby a computing device as “zooming in” on a particular portion of thedigital image. The computer vision techniques, as implemented by thecomputing device, may then determine whether the user is interested in aparticular object or characteristic. Object recognition techniques, forinstance, may be performed using a classifier as part of machinelearning to recognize an object in the portion when “zoomed in” and thusthe user is likely interested in the object. This may also be used todetermine a characteristic of the object. A user, for instance, may“zoom in” via a gesture such that the object is not recognizable, butcharacteristics of the object are such as material, color, pattern, andso forth. In this way, the computer vision techniques as implemented bya computing device may then determine that the user is interested in thecolor and not the object. In another example, both the object and thecharacteristic of the object may be inferred by the computing device.

This focus as inferred by the computing device using these computervision techniques increases accuracy of determining user interest overconventional techniques that rely on the digital image as a whole.Conventional techniques, for instance, may rely on tags that refer tocharacteristics of an entirety of the digital image, e.g., involvingeach of the objects and characteristics included in the digital image.The user, on the other hand, may not be interested in each of theseobjects. Accordingly, the techniques described herein may determine withan increased amount of resolution which objects and characteristics ofthe objects are a focus of the user interest through monitoring usermanipulation of the digital images. This may also be used to determinewhich object and characteristics of the object that are not of interestto the user. This may be based, for instance, on particular types ofobjects, colors, and so on that are included in the digital images butare not subject to this manipulation. From this, it may be inferred thatthe user is not interested in these types of objects or characteristics.As a result, a computing device that implements these techniquesexperiences increased operational efficiency in digital content control(e.g., accuracy in recommendations and digital marketing content) anduser inefficiency in interacting with this content. Further discussionof these and other examples is included in the Computer Vision and UserSegment section in the following and is illustrated in FIGS. 3-10 .

In a second example, techniques are described for surfacing listings ofproducts and services to client device users by attempting to determineproducts and services that these users are likely to want, and thensurfacing the determined products and services. Conventional systems maydetermine these products and services based on a variety of informationcollected about a user, such as search queries entered (by text, voice,and so on) by the user, web pages to which the user has navigated,content of personal communications (e.g., email, text messages, instantmessages, and so on), content posted on the user's social media profile,and so forth. However, such conventional systems fail to considercombinations of products depicted in a user's digital image, e.g., froma single digital image or part of a video. As a result, theseconventional techniques fail to account for how users combine productsand services acquired into a particular “look and feel”. Withoutconsidering this “look and feel,” conventional systems can surfaceproducts and services that do not match the actual tastes of users.

To overcome these problems, complete-the-look techniques are leveragedby a computing device in a digital medium environment. Thecomplete-the-look techniques are used by the computing device to processa digital image and from this identify products and services to surfaceto a user as digital content, e.g., via an e-commerce platform, asdigital marketing content, and so on. As part of this, thecomplete-the-look techniques are used by the computing device toinitially determine items in digital images, such as by using one ormore object recognition techniques. By way of example, thecomplete-the-look techniques may be used to determine items in a livestream of digital images (e.g., video) captured of a living room, suchas couches, lamps, side tables, and so on. The complete-the-looktechniques are then used to associate information with the digital imagethat is indicative of the detected items, e.g., text tags indicative ofthe detected items. For instance, the complete-the-look techniques mayassociate text tags with the video of the living room that areindicative of the detected items in the living room, e.g., metadata. Acatalog of this digital image and associated information can bemaintained in storage.

From the digital image and the associated information, thecomplete-the-look techniques are then used by the computing device todetermine common items in the digital image. By way of example, thecomplete-the-look techniques may be used to determine that rooms havingcouches, lamps, side tables, and so on, also typically includetelevisions. Given the determined common items, the complete-the-looktechniques are also able to determine items that are “missing” fromdigital images corresponding to certain contexts. Assume that thecomplete-the-look techniques are used to determine that digital imagescorresponding to living rooms do typically include couches, lamps, sidetables, and televisions. In this scenario, the complete-the-looktechnique as implemented by a computing device receives a digital imagefrom a user of a living room that depicts a couch, lamp, and side table,but not a television. Based on this, the complete-the-look techniquesare used to determine that a television is missing from the living room.To “complete-the-look,” the complete-the-look techniques can surfacetelevisions listed via one or more listing services to the user.

This may also be performed using machine-learning techniques. Amachine-learning model (e.g., a neural network), for instance, may betrained by the computing device using a plurality of training digitalimages. The plurality of training digital images depict physicalenvironments and items disposed in the physical environments. Thus, thetraining digital images depict “good” examples of physical environmentsand the items (e.g., objects) disposed in those environments. From this,the machine-learning module “learns” which items are typically includedin particular physical environments and/or in combination with otheritems. The trained model is then used to implement the missing itemtechniques.

A subsequent digital image, for instance, may be generated by a digitalcamera of a computing device as a “live feed” of digital images of aphysical environment. A determination is then made that an item ismissing in the subsequent physical environment based on the subsequentdigital image and the trained machine-learning model by the computingdevice.

In response, computing device may indicate which item is missing. In oneexample, the computing device locates AR digital content thatcorresponds to the item that is missing, e.g., as a listing of productsor services that are available for sale from the service provider systemand represented by the AR digital content. The located AR digitalcontent is then output as part of the live feed of digital images by thecomputing device. In this way, the missing items may be detected andsurfaced to a user automatically and without user intervention anddisplayed as appearing in a physical environment of a user, therebyimproving computational and user efficiency. Further discussion of theseand other examples is included in the Computer Vision and Missing Itemsection and is illustrated in FIGS. 11-14 .

In the following discussion, an example environment is first describedthat may employ the techniques described herein. Example procedures andsystems are also described and shown as blocks which may be performed inthe example environment as well as other environments. Consequently,performance of the example procedures is not limited to the exampleenvironment and systems and the example environment and systems are notlimited to performance of the example procedures.

Example Environment

FIG. 1 is an illustration of a digital medium environment 100 in anexample implementation that is operable to employ computer visiontechniques described herein. The illustrated environment 100 includes acomputing device 102 that is communicatively coupled to a serviceprovider system 104 via a network 106. Computing devices that implementthe computing device 102 and the service provider system 104 may beconfigured in a variety of ways.

A computing device, for instance, may be configured as a desktopcomputer, a laptop computer, a mobile device (e.g., assuming a handheldconfiguration such as a tablet or mobile phone), configured to be worn(e.g., as goggles as illustrated for computing device 102) and so forth.Thus, a computing device may range from full resource devices withsubstantial memory and processor resources (e.g., personal computers,game consoles) to a low-resource device with limited memory and/orprocessing resources (e.g., mobile devices). Additionally, although asingle computing device is shown, a computing device may berepresentative of a plurality of different devices, such as multipleservers utilized by a business to perform operations “over the cloud”for the service provider system 104 as described in FIG. 15 .

The computing device 102 is illustrated as being worn by a user 108 in aphysical environment 110, e.g., a living room. The computing device 102includes a digital camera 112 that is configured to capture digitalimages 114 of an outside physical environment (e.g., the living room),such as through use of a charge coupled device (CCD) sensor. Thecaptured digital images 114 may then be stored as pixels in acomputer-readable storage medium and/or rendered for display by adisplay device, e.g., LCD, OLED, LED, etc.

The computing device 102 also includes a camera platform manager module116 that is configured to implement and execute a camera platform 118(e.g., through use of a processing system and computer-readable storagemedia) that may serve as a basis for a variety of functionality. Thecamera platform 118, for instance, may implement a “live view” formed ofdigital images 114 taken of the physical environment of the computingdevice 102. These digital images 114 may then serve as a basis tosupport other functionality.

An example of this functionality is illustrated as an object inventorymanager module 120. The object inventory manager module 120 isrepresentative of functionality to manage an inventory of objects. Thismay include objects that are owned by the user 108 and/or objects thatare desired by the user 108, e.g., for purchase. This may be implementedby the object inventory manager module 120 through use of the cameraplatform 118 in a variety of ways.

In a first such example, the object inventory manager module 120 isconfigured to collect digital images 114. This may include digitalimages 114 of physical objects in the living room in this example ordigital images captured of physical photos, e.g., from a magazine, apicture taken of a television screen or other display device, and so on.The digital images 114 may also be captured of a user interface outputby the computing device 102, e.g., as a screenshot from a frame buffer.

The object inventory manager module 120 includes object recognitionfunctionality to recognize objects included within the digital images114, e.g., via machine learning. From this, the object inventory managermodule 120 may collect data pertaining to this recognition. Datadescribing the recognized objects, for instance, may be communicated viathe network 106 to the service provider system 104. The service providersystem 104 includes a service manager module 122 that is configured toobtain data related to the objects (e.g., through use of a search) froma storage device 124. This data may then be communicated back to thecomputing device 102 via the network 106 for use by the object inventorymanager module 120.

The object inventory manager module 120, for instance, may generateaugmented reality digital content 126 (illustrated as stored in astorage device 128) for output in the user interface of the computingdevice 102 as part of a “live feed” of digital images taken of thephysical environment, e.g., the living room. The AR digital content 126may describe characteristics of the object, a brand name of the object,a price for which the object is available for sale or purchase (e.g.,via an online auction), and so forth. This AR digital content 126 isthen displayed proximal to the object by the object inventory managermodule 120. In this way, the camera platform supports functionality forthe user 108 to “look around” the physical environment 110 and providesadditional information and insight into characteristics of objectsincluded within the physical environment 110.

In another example, the object inventory manager module 120 leveragesthe camera platform 118 to make recommendations for a user. The digitalimages 114, for instance, may also be processed by the object inventorymanager module 120 using object recognition as implemented using machinelearning. In this example, the digital images are used to generate aprofile (e.g., a user profile) based on characteristics learned from thedigital images 114. This profile is then used as a basis to formrecommendations (e.g., through machine learning), such as to configuredigital marketing content having product suggestions based on thesecharacteristics.

The profile, for instance, may include digital images taken of the user108. From this, the object inventory manager module 120 may determinecharacteristics and tastes of the user 108. Digital images may also becollected from sources that do not include the user but are desired bythe user, e.g., of other humans in person, from physical photos, and soforth. From this, the object inventory manager module 120 may generaterecommendations based on the user profile. In this way, the objectrecognition module may increase accuracy and as a result increasecomputational efficiency in generation of recommendations based on thecamera platform 118.

FIG. 2 depicts a system 200 in an example implementation showingoperation of the camera platform manager module 116 of FIG. 1 in greaterdetail. The following discussion describes techniques that may beimplemented utilizing the previously described systems and devices.Aspects of the procedure as shown stepwise by the modules of FIG. 2 maybe implemented in hardware, firmware, software, or a combinationthereof. The procedure is shown as a set of blocks that specifyoperations performed by one or more devices and are not necessarilylimited to the orders shown for performing the operations by therespective blocks.

To begin, a digital image 114 is obtained by the camera platform managermodule 116. The digital image 114, for instance, may be captured using adigital camera 112, as a screenshot captured from a frame buffer of thecomputing device 102, and so forth. The digital image 114 is thenprocessed by an object recognition module 202 to recognize an objectwithin the digital image 114 or characteristics of the object, e.g.,material, color, pattern, and so forth. The object recognition module202, for instance, may employ a machine-learning module 204 configuredto employ models 206 as a classifier usable to recognize the objectusing machine learning, e.g., neural networks, convolutional neuralnetworks, deep learning networks, structured vector machines, decisiontrees, and so forth. The models 206, for instance, may be trained usingtraining digital images that are tagged with correspondingidentifications of the objects and/or characteristics of the objects.

In an implementation, these training digital images and tags areobtained from a commerce service provider system that are tagged bysellers using the system. As a result, a multitude of accurately taggedtraining digital images may be obtained with minimal computation anduser cost as opposed to conventional manual tagging techniques. Althoughillustrated as implemented locally by the computing device 102, thisfunctionality may also be implemented in whole or in part by a serviceprovider system 104 via the network 106.

Thus, the object recognition data 208 describes an object included inthe digital image 114 and/or characteristics of the object. An objectdata collection module 210 is then employed to collect object metadata212 that pertains to the recognized object or characteristics of theobject. This may be performed locally through a search of a localstorage device and/or remotely through interaction with a servicemanager module 122 of a service provider system 104 via a network 106.

A variety of different types of object metadata 212 may be obtained froma variety of different types of service provider systems 104. In oneexample, the service provider system 104 provides information relatingto purchase or sale of the object, e.g., product name, productdescription, price for purchase or sale (e.g., based on onlineauctions), and so forth. In another example, the service provider system104 provides information relating to customer reviews of the product,e.g., a number of “stars” or other rating, textual reviews, and soforth. In a further example, the metadata describes replacement parts ofthe object, e.g., filters, batteries, bulbs, and so forth. The objectmetadata 212 in this instance may be used to then order thesereplacement parts in an efficient and intuitive manner, e.g., throughselection of AR digital content formed from the metadata.

The object metadata 212 in this example is then provided to an augmentedreality (AR) configuration module 214. The AR configuration module 214,for instance, may be configured to generate AR digital content 126 fromthe object metadata 212 for display proximal to the object by an ARrendering module 216 to an output device 218, e.g., display device,audio output device, tactile output device, and so forth. The augmentedreality content in this example includes both content supported alongwith a direct view of a physical environment and content supported alongwith a recreated view of the physical environment. In this way, throughuse of a camera platform 118 as implemented by the camera platformmanager module 116, a user may simply “look around” using a live feed ofdigital images 114, select objects in the digital images 114, and obtainmetadata related to the object.

In the replacement part example, object recognition may be used to firstidentify an object. This identification may then be used as a “look up”to locate replacement parts associated with the object, e.g., filters,bulbs, batteries, and so forth. AR digital content may then be outputthat is selectable to purchase these items in a direct view in the userinterface. In an example, this information is correlated with a pastpurchase history, such that the AR digital content may indicate “when”to replace the replacement part, when the replacement part was lastpurchased, when it is due to be replaced, and so forth. This may also beused to monitor user interaction to assign users to segments of a userpopulation and for missing item determination, further discussion ofwhich is included in corresponding sections in the followingdescription.

In general, functionality, features, and concepts described in relationto the examples above and below may be employed in the context of theexample procedures described in this section. Further, functionality,features, and concepts described in relation to different figures andexamples in this document may be interchanged among one another and arenot limited to implementation in the context of a particular figure orprocedure. Moreover, blocks associated with different representativeprocedures and corresponding figures herein may be applied togetherand/or combined in different ways. Thus, individual functionality,features, and concepts described in relation to different exampleenvironments, devices, components, figures, and procedures herein may beused in any suitable combinations and are not limited to the particularcombinations represented by the enumerated examples in this description.

Computer Vision and User Segment

FIG. 3 depicts a system 300 in an example implementation showingoperation of the camera platform manager module 116 of FIG. 1 in greaterdetail as employing a user profile. FIG. 4 depicts an example system 400of obtaining a recommendation based on the generated user profile ofFIG. 3 . FIG. 5 depicts a procedure 500 in an example implementation inwhich a user is assigned to a user segment used to generate arecommendation based on monitored user interaction with digital images.FIG. 6 depicts a procedure 600 in which monitored manipulation of a userwith a digital image as focusing on an object or characteristic is usedto assign the user to a user segment. FIGS. 7-10 depict user interfacesshowing examples 700, 800, 900, 1000 of user manipulation and focus.

The following discussion describes techniques that may be implementedutilizing the previously described systems and devices. Aspects of theprocedure as shown stepwise by the modules of FIGS. 3 and 4 may beimplemented in hardware, firmware, software, or a combination thereof.The procedure is shown as a set of blocks that specify operationsperformed by one or more devices and are not necessarily limited to theorders shown for performing the operations by the respective blocks. Inportions of the following discussion, reference will be made to FIGS.3-10 .

In this example, the object inventory manager module 120 leverages thecamera platform 118 to control output of digital content to a user,e.g., make recommendations for a user 108, digital marketing content(i.e., ads), and so forth. The digital images 114, for instance, mayalso be processed by the object inventory manager module 120 usingobject recognition as implemented using machine learning. The digitalimages 114 are used to generate a profile (e.g., a user profile 302)based on characteristics learned from the digital images 114. This userprofile 302 is then used as a basis to form recommendations (e.g.,through machine learning as further described in relation to FIG. 4 ),such as to configure digital marketing content having productsuggestions based on these characteristics.

To do so, object recognition data 208 is generated as previouslydescribed using an object recognition module 202 and machine learning asimplemented by a machine-learning module 204 and associated model 206,e.g., as a classifier. The object recognition data 208 is then providedto a profile generation module 304 to generate the user profile 302.

The object recognition data 208 may be used to identify a type ofobject, e.g., running shoes, cars, etc. The object recognition data 208may also be used to describe characteristics of the objects, e.g., apreference for “clean” uncluttered objects, types of patterns, types ofmaterials, textures, and so on that are learned from the digital images.This may be used, for instance, to infer interests of “modern, clean,uncluttered” lines versus ornate, baroque lines. This may be performedin a variety of ways. In one example, the profile generation module 304generates the user profile 302 as a set of rules to specify anassociation of user preferences with object characteristics 306. Inanother example, machine learning is employed, such as through the userof neural networks as part of classifiers which output a probabilitythat the object is or is not a particular type of object, has a certaincharacteristic, and so forth. Machine learning may also be used todetermine hidden states and associations of object characteristics 306and user preferences.

The object recognition data 208, for instance, may be collected fromwebpages navigated to by a user, e.g., from a service provider system104. This is therefore used to monitor user interaction “in thebackground” with the webpages and content of the webpages. Associatedactions taken by the user to manipulate the digital image, e.g., to“click on” a digital image on the webpage, “zoom in,” and so forth maythen be incorporated as part of a loss function to learn the userpreferences, e.g., to adjust weights and/or connections within a neuralnetwork. In an implementation, this is performed across differentapplications, webpages, and so forth, with which, a user has interactedusing the computing device 102.

From this, the object inventory manager module 120 may generaterecommendations based on the user profile 302. In this way, the objectrecognition module may increase accuracy and as a result increasecomputational efficiency in generation of recommendations based on thecamera platform 118.

The user profile 302 may be formed based on digital images thatoriginate from a variety of different sources. A user, for instance, maytake a multitude of “selfies” and thus readily identify preferences ofthe user as related to the user, themselves. In this way, the selfiesprovide additional insight and thus accuracy of a model trained usingmachine learning regarding user preferences.

In another instance, this may be used to determine user preferences forparticular types of products. The object recognition data 208, forinstance, may describe a “type” of object captured from the digitalimages 114. Object characteristics 306 learned from the objectrecognition data 208 may then be used to train a model to associatethose characteristics with particular types of products, e.g., red cars,black t-shirts, red phone cases, and so forth. This may also be usedacross product types, such as to continue with the previous example inwhich a user's preference for red cars and phone cases indicates ageneral overall preference for red products. In an implementation, aweighting may also be learned to combine product-type models with modelsused across product types to address overall user preferences withspecific preferences regarding a particular product type.

FIG. 4 depicts an example of generation of the user profile 302 ingreater detail. To begin, a digital image 114 is obtained by the cameraplatform manager module 116 as before. The digital image 114, forinstance, may be captured using a digital camera 112, as a screenshotcaptured from a frame buffer of the computing device 102, a selfie,taken of a physical image from a physical medium, and so forth.

The digital image 114 is then processed by an object recognition module202 to recognize an object and/or characteristics of the object withinthe digital image 114. The object recognition module 202, for instance,may employ a machine learning module 204 configured to employ models 206usable to recognize the object using machine learning, e.g., neuralnetworks. The models 206, for instance, may be trained as classifiersusing training digital images that are tagged with correspondingidentifications. In an implementation, these training digital images andtags are obtained from a commerce service provider system that aretagged by sellers using the system. As a result, a multitude of trainingdigital images may be obtained with minimal computation and user cost asopposed to conventional manual tagging techniques. Although illustratedas implemented locally by the computing device 102, this functionalitymay also be implemented in whole or in part by a service provider system104 via the network 106.

Thus, the object recognition data 208 describes an object and/orcharacteristics of the object included in the digital image 114. Thedigital image 114 may also be captured of other users that havefavorable characteristics, e.g., of a person on a street having adesired jacket, a digital image taken of a physical photograph of apopstar in a physical magazine, and so forth. Thus, this objectrecognition data 208 is collected to generate user profile data thatdescribes the user and/or characteristics of other users as recognizedfrom the digital image 114.

As shown in FIG. 4 , the user profile 302 may then be communicated tothe service provider system 104, which forms a recommendation 402 basedon the user profile 302. The recommendation 402, for instance, may begenerated using machine learning based on a user segment, to which, theuser belongs as identified through non-negative matrix factorization. Inthis way, the camera platform 118 of the object inventory manager module120 may address likely user desires based on the digital images andobject recognition supported by the platform. Although described asimplemented by the computing device 102, this functionality may also beimplemented in whole or in part by the service provider system 104,e.g., in response to communicated images as part of a social networkservice.

FIG. 5 depicts a procedure 500 in an example implementation in which auser is assigned to a user segment used to generate a recommendationbased on monitored user interaction with digital images. To begin, userinteraction is monitored with respect to a plurality of items of digitalcontent (block 502). The camera platform manager module 116, forinstance, may monitor webpages, screens of a user interface output by anapplication executed by the computing device 102, and so on.

Digital images are identified within the plurality of items of digitalcontent (block 504) that are subject of the user interaction. The cameraplatform manager module 116, for instance, may detect which digitalimages of a plurality of digital images within an item of digitalcontent are selected (e.g., “clicked”), subject to the hovering of acursor over the image, a gesture (e.g., to “zoom in”), spoken utterance,and so forth. Thus, the user interaction in this example helps todetermine user interest in respective images by focusing on particularportions of the image.

Objects are recognized that are included within the identified digitalimages. Characteristics of the recognized objects are also recognized(block 506). The object recognition module 202, for instance, may beused to generate object recognition data 208 that identifies whichobjects are included in the digital images 114. This may also be used todescribe object characteristics 306, such as patterns, textures,material properties, surface treatments of the materials (e.g., rusted,“industrial”), and so forth

A user corresponding to the user interaction is assigned to a usersegment of a user population. The assignment is based on the recognizedobjects, the recognized characteristics, and the corresponding userinteraction (block 508). This may be used, for instance, to generate amodel using machine learning. The model is trained based on this data inwhich the user interaction is used as part of a loss function to trainthe model based on corresponding user actions undertaken as part of theimage. The user interactions, for instance, may indicate a userpreference for certain colors, textures, materials, and so forth andthus the model may be used to reflect these user preference as part ofgenerating a recommendation. This may also be used to indicate objectsor characteristics that the user is not interested in, e.g., included inthe digital images but are not subject to user manipulation as focusingon these objects or characteristics.

The object recognition data 208, for instance, may be used to definemembership of the user within a particular segment of a user population,e.g., for digital marketing purposes. A recommendation is then generatedby the computing device 102 based on the assigned user segment (block510). Output is then controlled by the computing device of a subsequentitem of digital content based on the generated recommendation (block512), e.g., of an item of digital marketing content configured to causeconversion of a product or service. In this way, the recommendation isgenerated with increased accuracy over conventional techniques, whichimproves operation of a computing device that generates theserecommendations.

FIG. 6 depicts a procedure 600 in which monitored manipulation of a userwith a digital image as focusing on an object or characteristic is usedto assign the user to a user segment. Manipulation by the user of adigital image via a user interface is monitored as focusing on an objector characteristic of the object in the digital image (block 602). Acamera platform manager module 116, for instance, may monitor useinteraction with a digital image output using a display device of thecomputing device 102. This interaction may occur in a variety of ways,such as a gesture (e.g., zoom in, zoom out, tap), cursor control deviceand keyboard (e.g., control button keypress and a scroll wheel), aspoken utterance, and so forth.

As shown in the example 700 of FIG. 7 , for instance, a digital image702 is displayed in a user interface of a display device of thecomputing device 102. The digital image 702 includes a variety ofdifferent products, which are clothing items in this example. Inconventional techniques, a user's interest was specified for the digitalimage as a whole, which lacks accuracy because some objects in thedigital image and even characteristics of the objects may be of interestwhile others may not.

In the illustrated example, however, manipulation by a user via userinput received via the display device (e.g., using touchscreenfunctionality) is monitored to determine a focus of a user with respectto a particular portion of the digital image. A result of this focus isdepicted in the example 800 of FIG. 8 in which a user has “zoomed in” toview a particular item of clothing in a corresponding portion 802 of thedigital image 702 of FIG. 7 .

This manipulation causes the camera platform manager module 116 toinitiate object recognition techniques by an object recognition module202 of the camera platform manager module 116 to identify an objectand/or characteristics of the object in this portion of the digitalimage as previously described. In this way, the profile generationmodule 304 may generate a user profile to reflect this interest, and mayalso reflect disinterest in other objects and/or characteristics in thedigital image 702 that are not subject to this manipulation, e.g., otherclothing items, colors of the clothing items, and so forth.

This may also be used to distinguish between interest in the object andinterest in a characteristic of the object. As shown in the example 900of FIG. 9 , for instance, a digital image 902 is again output in a userinterface of the computing device that includes multiple objects, e.g.,items of clothing Manipulation of the digital image 902 in response to auser input causes the computing device 102 to zoom in to a portion 1002of the digital image 902 as shown in FIG. 10 . In this example, theportion 1002 is not sufficient to recognize the corresponding objectusing object recognition techniques. However, the portion is sufficientto recognize characteristics of the object, e.g., a color, pattern,material, and so forth. In this example, the user interest is focused onthe characteristic, and not the object. This focus is used as part ofthe user profile 302 to assign a user to a corresponding segment asfurther described below. Other examples are also contemplated in whichthe object and characteristic are both employed for the assignmentand/or generation of the user profile 302.

The user is then assigned to a user segment of a user population by theobject inventory manager module 120. The assignment is based on themanipulation as focusing on the object or characteristic of the object(block 604) as described above. This assignment is then used to generatea recommendation (block 606) and control output of digital content(block 608) as described above. In this way, the camera platform managermodule 116 supports increased accuracy and computational efficiency ascompared with conventional digital content control techniques.

Computer Vision and Missing Item Determination

FIG. 11 depicts images of an example implementation in which items thatare common to a particular context are determined from the images toenable a further determination of items that are missing from anotherimage of the particular context. FIG. 12 depicts a scenario in anexample implementation in which augmented and/or virtual reality (AR/VR)techniques are used to suggest items that are missing from a particularcontext. FIG. 13 depicts another scenario in an example implementationin which AR/VR techniques are used to suggest different items that aremissing from the particular context. FIG. 14 depicts a procedure in anexample implementation in which machine-learning techniques and ARdigital content are employed as part of missing item determination.

The following discussion describes techniques that may be implementedutilizing the previously described systems and devices. Aspects of theprocedure as shown stepwise may be implemented in hardware, firmware,software, or a combination thereof. The procedure is shown as a set ofblocks that specify operations performed by one or more devices and arenot necessarily limited to the orders shown for performing theoperations by the respective blocks. In portions of the followingdiscussion, reference will be made to FIGS. 11-15 .

Techniques are described as implemented by the camera platform managermodule 116 for surfacing listings of products and services to users inan attempt to determine products and services that these users arelikely to want, and then surface the determined products and services.Conventional systems may determine these products and services based ona variety of information collected about a user, such as search queriesentered (by text, voice, and so on) by the user, web pages to which theuser has navigated, content of personal communications (e.g., email,text messages, instant messages, and so on), content posted on theuser's social media profile, and so forth. However, such conventionalsystems fail to consider combinations of products depicted in a user'sdigital image, e.g., from a single digital image or video. In this way,these conventional techniques fail to account for how users combineproducts and services acquired into a particular “look and feel.”Without considering this “look and feel,” conventional systems cansurface products and services that do not match the actual tastes ofusers.

To overcome these problems, complete-the-look techniques are leveragedin a digital medium environment. The complete-the-look techniques areimplemented to consider a digital image and identify products andservices to surface to a user, e.g., via an e-commerce platform which isthen surfaced as AR digital content in a live feed of digital images114.

As part of this, the complete-the-look techniques are used by acomputing device to determine which items are included in a digitalimage, such as by using one or more object recognition techniques aspreviously described by the object recognition module 202. By way ofexample, the complete-the-look techniques as implemented by thecomputing device 102 determines items in a video captured of a livingroom, such as couches, lamps, side tables, and so on. The computingdevice 102 then associates information with the digital image 114 thatis indicative of the detected items, e.g., text tags indicative of thedetected items. For instance, the computing device 102 associates texttags with the video of the living room that are indicative of thedetected items in the living room, e.g., metadata. A catalog of thisdigital image and associated information can be maintained in a storagedevice.

From the digital image and the associated information, the cameraplatform manager module 116 determines common items in the digitalimage. By way of example, the camera platform manager module 116determines that rooms having couches, lamps, side tables, and so on,also typically include televisions. Given the determined common items,the camera platform manager module 116 is also able to determine itemsthat are “missing” from digital image corresponding to certain contexts.Assume that the camera platform manager module 116 determines thatdigital images corresponding to living rooms do typically includedcouches, lamps, side tables, and televisions. In this scenario, thecamera platform manager module 116 receives a digital image 114 from auser of a living room that depicts a couch, lamp, and side table, butnot a television. Based on this, the camera platform manager module 116determines that a television is missing from the living room. To“complete-the-look,” the camera platform manager module 116 surfacestelevisions listed via one or more listing services to the user usingdigital content, e.g., as AR digital content 126.

In this context, consider FIG. 11 , which depicts example images of afirst, second, third, and fourth living room. In this example 1100, thecamera platform manager module 116 detects that the images of the first,second, and third living rooms each depict a television—as indicated bythe dashed ellipse. These images may be cataloged and maintained in ahistorical data repository by the camera platform manager module 116,e.g., for comparison to future received images. The fourth living roomimage may be received by the complete-the-look system in connection witha user. For instance, the user may upload the digital image 114 to thecamera platform manager module 116 with a request to process the digitalimage 114 to determine other items to “complete-the-look” in relation tothe living room. Alternately or in addition, the digital image 114 maybe scraped from a social media profile of the user, a photo sharing webpage of the user, an email of the user, and so forth. In either case,the camera platform manager module 116 determines that the fourth livingroom image does not depict a television, based on object recognition andcomparison of recognized objects to the “common” objects of livingrooms. Based on this, the camera platform manager module 116 surfacesthe missing items to the user, e.g., through communication with theservice manager module 122 of the service provider system 104.

The camera platform manager module 116 may surface determined items in avariety of ways. By way of example, the camera platform manager module116 can surface a user interface that includes a plurality of listedtelevisions, which the user can select, e.g., for purchase. Alternatelyor in addition, the camera platform manager module 116 can surface oneor more determined items using augmented and/or virtual reality (AR/VR)techniques as part of a live feed of digital images 114 as described inrelation to FIG. 2 .

In this context, consider FIGS. 12 and 13 . FIGS. 12 and 13 depictexample scenarios at 1200, 1300 in which the camera platform managermodule 116 leverages AR/VR to surface determined “missing” items forsuggestion to a user. In particular, these figures represent scenariosin which a user is standing in the fourth living room with a computingdevice 102 that captures a live feed of digital images 114 of the fourthliving room. In the illustrated examples 1200, 1300, the computingdevice 102 is shown presenting video content of the fourth living roomvia a display device 1202. In these examples, however, the computingdevice 102 is also shown presenting superimposed suggested items in theliving room using AR digital content 126.

In the context of FIG. 12 , the complete-the-look technique providesdata to the computing device 102 so that it can superimpose a television(e.g., the determined missing item) on the living room wall. In thecontext of FIG. 13 , the complete-the-look technique is used to providedata to superimpose framed pictures (e.g., the determined missing items)on the living room wall via the display device 1202.

It is to be appreciated that the complete-the-look techniques arecapable of completing looks in a variety of different scenarios withoutdeparting from the spirit or scope of the techniques described herein.For instance, the complete-the-look techniques can be used to complete aperson's outfit. By way of example, the complete-the-look techniques canbe used by the computing device 102 to determine that a person indigital image is wearing a particular type of top and particular bottomsusing object recognition techniques. The complete-the-look techniquesmay also be used to determine that other clothing items or fashionaccessories are missing from the outfit based on other digital imagedepicting persons wearing a similar top and similar bottoms.

It is also to be appreciated that the complete-the-look techniques maydetermine attributes of items in addition to simply recognizing theseitems. Rather than simply detect a couch and associate a text tag for“couch” with digital image, for instance, the complete-the-looktechniques may be used by the computing device 102 to detect that thecouch is a leather couch having a particular style and associate theseattributes with the couch. In this way, the complete-the-look techniquesas implemented by a computing device can determine particular items thatare missing as having certain attributes from other digital imagesdepicting similar styles.

In one or more implementations, the complete-the-look techniques mayalso consider a location associated with a user (e.g., the physicalenvironment captured by the digital image 114) when determining missingitems and items to surface to a user. By way of example, thecomplete-the-look techniques may be used to surface different items to auser determined to be located in Kansas that are surfaced to a userdetermined to be located in the Netherlands. In other words, thecomplete-the-look techniques are used to consider trends (e.g., fashion,interior decorating, toys, and so on) in different locations. Thecomplete-the-look techniques may also be forward looking, such that iftrends at a location typically follow trends at a different location(e.g., fashion trends in New York follow fashion trends from Paris a fewmonths later), the complete-the-look techniques can surface items tousers in the location based on the trends at the different, followedlocation.

The complete-the-look techniques can also be leveraged to aid users tonavigate through digital catalogs of listed products and services. Forinstance, the complete-the-look techniques enables a user to takemultiple digital images of an item (e.g., an engine block) to determineone or more items in which a user is interested. For instance, thecomplete-the-look techniques can process the captured digital image todetect which item of an engine block is missing, which item of theengine block can be upgraded, which item of the engine block can bereplaced, and so forth.

FIG. 14 depicts a procedure 1400 in an example implementation in whichmachine-learning techniques and AR digital content are employed as partof missing item determination. To begin in this example, amachine-learning model (e.g., a neural network) is trained by thecomputing device 102 using a plurality of training digital images. Theplurality of training digital images depict physical environments anditems disposed in the physical environments (block 1402). Thus, thetraining digital images depict “good” examples of physical environmentsand the items (e.g., objects) disposed in those environments. Thetrained model is then used to implement the missing item techniquesdescribed above.

A subsequent digital image 114, for instance, may be received by thecamera platform manager module 116 from the digital camera 112 as a“live feed” of digital images of a physical environment (block 1404),such as of the fourth living room of FIG. 13 . A determination is thenmade that an item is missing in the subsequent physical environmentbased on the subsequent digital image and the trained machine-learningmodel (block 1406) by the computing device 102. This may be performedlocally at the computing device 102 and/or remotely by the serviceprovider system 104.

In response, the camera platform manager module 116 locates AR digitalcontent that corresponds to the item that is missing (block 1408), e.g.,as a listing of products or services that are available for sale fromthe service provider system 104 and represented by the AR digitalcontent. The located AR digital content is then output as part of thelive feed of digital images (block 1401) by a corresponding displaydevice 1202 of the computing device 102. In this way, the missing itemsmay be detected and surfaced to a user automatically and without userintervention, thereby improving computational and user efficiency.

Example System and Device

FIG. 15 illustrates an example system generally at 1500 that includes anexample computing device 1502 that is representative of one or morecomputing systems and/or devices that may implement the varioustechniques described herein. This is illustrated through inclusion ofthe camera platform manager module 116. The computing device 1502 maybe, for example, a server of a service provider, a device associatedwith a client (e.g., a client device), an on-chip system, and/or anyother suitable computing device or computing system.

The example computing device 1502 as illustrated includes a processingsystem 1504, one or more computer-readable media 1506, and one or moreI/O interface 1508 that are communicatively coupled, one to another.Although not shown, the computing device 1502 may further include asystem bus or other data and command transfer system that couples thevarious components, one to another. A system bus can include any one orcombination of different bus structures, such as a memory bus or memorycontroller, a peripheral bus, a universal serial bus, and/or a processoror local bus that utilizes any of a variety of bus architectures. Avariety of other examples are also contemplated, such as control anddata lines.

The processing system 1504 is representative of functionality to performone or more operations using hardware. Accordingly, the processingsystem 1504 is illustrated as including hardware element 1510 that maybe configured as processors, functional blocks, and so forth. This mayinclude implementation in hardware as an application specific integratedcircuit or other logic device formed using one or more semiconductors.The hardware elements 1510 are not limited by the materials from whichthey are formed or the processing mechanisms employed therein. Forexample, processors may be comprised of semiconductor(s) and/ortransistors (e.g., electronic integrated circuits (ICs)). In such acontext, processor-executable instructions may beelectronically-executable instructions.

The computer-readable storage media 1506 is illustrated as includingmemory/storage 1512. The memory/storage 1512 represents memory/storagecapacity associated with one or more computer-readable media. Thememory/storage 1512 component may include volatile media (such as randomaccess memory (RAM)) and/or nonvolatile media (such as read only memory(ROM), Flash memory, optical disks, magnetic disks, and so forth). Thememory/storage 1512 component may include fixed media (e.g., RAM, ROM, afixed hard drive, and so on) as well as removable media (e.g., Flashmemory, a removable hard drive, an optical disc, and so forth). Thecomputer-readable media 1506 may be configured in a variety of otherways as further described below.

Input/output interface(s) 1508 are representative of functionality toallow a user to enter commands and information to computing device 1502,and also allow information to be presented to the user and/or othercomponents or devices using various input/output devices. Examples ofinput devices include a keyboard, a cursor control device (e.g., amouse), a microphone, a scanner, touch functionality (e.g., capacitiveor other sensors that are configured to detect physical touch), a camera(e.g., which may employ visible or non-visible wavelengths such asinfrared frequencies to recognize movement as gestures that do notinvolve touch), and so forth. Examples of output devices include adisplay device (e.g., a monitor or projector), speakers, a printer, anetwork card, tactile-response device, and so forth. Thus, the computingdevice 1502 may be configured in a variety of ways as further describedbelow to support user interaction.

Various techniques may be described herein in the general context ofsoftware, hardware elements, or program modules. Generally, such modulesinclude routines, programs, objects, elements, components, datastructures, and so forth that perform particular tasks or implementparticular abstract data types. The terms “module,” “functionality,” and“component” as used herein generally represent software, firmware,hardware, or a combination thereof. The features of the techniquesdescribed herein are platform-independent, meaning that the techniquesmay be implemented on a variety of commercial computing platforms havinga variety of processors.

An implementation of the described modules and techniques may be storedon or transmitted across some form of computer-readable media. Thecomputer-readable media may include a variety of media that may beaccessed by the computing device 1502. By way of example, and notlimitation, computer-readable media may include “computer-readablestorage media” and “computer-readable signal media.”

“Computer-readable storage media” may refer to media and/or devices thatenable persistent and/or non-transitory storage of information incontrast to mere signal transmission, carrier waves, or signals per se.Thus, computer-readable storage media refers to non-signal bearingmedia. The computer-readable storage media includes hardware such asvolatile and non-volatile, removable and non-removable media and/orstorage devices implemented in a method or technology suitable forstorage of information such as computer readable instructions, datastructures, program modules, logic elements/circuits, or other data.Examples of computer-readable storage media may include, but are notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, digital versatile disks (DVD) or other optical storage, harddisks, magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or other storage device, tangible media, orarticle of manufacture suitable to store the desired information andwhich may be accessed by a computer.

“Computer-readable signal media” may refer to a signal-bearing mediumthat is configured to transmit instructions to the hardware of thecomputing device 1502, such as via a network. Signal media typically mayembody computer readable instructions, data structures, program modules,or other data in a modulated data signal, such as carrier waves, datasignals, or other transport mechanism. Signal media also include anyinformation delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media include wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 1510 and computer-readablemedia 1506 are representative of modules, programmable device logicand/or fixed device logic implemented in a hardware form that may beemployed in some embodiments to implement at least some aspects of thetechniques described herein, such as to perform one or moreinstructions. Hardware may include components of an integrated circuitor on-chip system, an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), a complex programmable logicdevice (CPLD), and other implementations in silicon or other hardware.In this context, hardware may operate as a processing device thatperforms program tasks defined by instructions and/or logic embodied bythe hardware as well as a hardware utilized to store instructions forexecution, e.g., the computer-readable storage media describedpreviously.

Combinations of the foregoing may also be employed to implement varioustechniques described herein. Accordingly, software, hardware, orexecutable modules may be implemented as one or more instructions and/orlogic embodied on some form of computer-readable storage media and/or byone or more hardware elements 1510. The computing device 1502 may beconfigured to implement particular instructions and/or functionscorresponding to the software and/or hardware modules. Accordingly,implementation of a module that is executable by the computing device1502 as software may be achieved at least partially in hardware, e.g.,through use of computer-readable storage media and/or hardware elements1510 of the processing system 1504. The instructions and/or functionsmay be executable/operable by one or more articles of manufacture (forexample, one or more computing devices 1502 and/or processing systems1504) to implement techniques, modules, and examples described herein.

The techniques described herein may be supported by variousconfigurations of the computing device 1502 and are not limited to thespecific examples of the techniques described herein. This functionalitymay also be implemented all or in part through use of a distributedsystem, such as over a “cloud” 1514 via a platform 1516 as describedbelow.

The cloud 1514 includes and/or is representative of a platform 1516 forresources 1518. The platform 1516 abstracts underlying functionality ofhardware (e.g., servers) and software resources of the cloud 1514. Theresources 1518 may include applications and/or data that can be utilizedwhile computer processing is executed on servers that are remote fromthe computing device 1502. Resources 1518 can also include servicesprovided over the Internet and/or through a subscriber network, such asa cellular or Wi-Fi network.

The platform 1516 may abstract resources and functions to connect thecomputing device 1502 with other computing devices. The platform 1516may also serve to abstract scaling of resources to provide acorresponding level of scale to encountered demand for the resources1518 that are implemented via the platform 1516. Accordingly, in aninterconnected device embodiment, implementation of functionalitydescribed herein may be distributed throughout the system 1500. Forexample, the functionality may be implemented in part on the computingdevice 1502 as well as via the platform 1516 that abstracts thefunctionality of the cloud 1514.

CONCLUSION

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention defined in the appended claims is not necessarilylimited to the specific features or acts described. Rather, the specificfeatures and acts are disclosed as example forms of implementing theclaimed invention.

What is claimed is:
 1. A method comprising: training, by at least onecomputing device, a machine-learning model using a plurality of digitalimages depicting persons wearing outfits that include at least oneclothing item; receiving, by the at least one computing device, anadditional digital image subsequent to training the machine-learningmodel, the additional digital image being included in a live feed ofdigital images capturing an outfit of a person; determining, by the atleast one computing device, that at least one common clothing item inthe digital images of similar outfits to the outfit is missing from theoutfit of the person based on the additional digital image and thetrained machine-learning model; locating, by the at least one computingdevice, augmented reality digital content that corresponds to a missingclothing item of the outfit of the person; and outputting, by the atleast one computing device, the located augmented reality digitalcontent as part of the live feed of digital images.
 2. The method ofclaim 1, wherein the outfit and the similar outfits include a similartop.
 3. The method of claim 1, wherein the outfit and the similaroutfits include similar bottoms.
 4. The method of claim 1, furthercomprising: determining that at least one common fashion accessory inthe digital images of similar outfits to the outfit is missing from theoutfit of the person based on the additional digital image and thetrained machine-learning model; locating additional augmented realitydigital content that corresponds to a missing fashion accessory for theoutfit of the person; and outputting the additional augmented realitydigital content as part of the live feed of digital images.
 5. Themethod of claim 1, further comprising locating the augmented realitydigital content that corresponds to a missing clothing item based on astyle of the outfit of the person.
 6. The method of claim 1, furthercomprising locating the augmented reality digital content thatcorresponds to a missing clothing item based on a similar style to theoutfit of the person.
 7. The method of claim 1, wherein the locatedaugmented reality digital content is selectable to purchase the missingitem.
 8. The method of claim 1, wherein the located augmented realitydigital content is a listing for the missing item.
 9. The method ofclaim 1, wherein the plurality of digital images are training digitalimages.
 10. A system comprising: one or more processors; and memorystoring instructions that, responsive to execution by the one or moreprocessors, cause the one or more processors to perform operationsincluding: training a machine-learning model using a plurality ofdigital images depicting persons wearing outfits that include at leastone clothing item; receiving an additional digital image subsequent totraining the machine-learning model, the additional digital image beingincluded in a live feed of digital images capturing an outfit of aperson; determining that at least one common clothing item in thedigital images of similar outfits to the outfit is missing from theoutfit of the person based on the additional digital image and thetrained machine-learning model; locating augmented reality digitalcontent that corresponds to a missing clothing item of the outfit of theperson; and outputting the located augmented reality digital content aspart of the live feed of digital images.
 11. The system of claim 10,wherein the outfit and the similar outfits include a similar top. 12.The system of claim 10, wherein the outfit and the similar outfitsinclude similar bottoms.
 13. The system of claim 10, wherein theoperations further include: determining that at least one common fashionaccessory in the digital images of similar outfits to the outfit ismissing from the outfit of the person based on the additional digitalimage and the trained machine-learning model; locating additionalaugmented reality digital content that corresponds to a missing fashionaccessory for the outfit of the person; and outputting the additionalaugmented reality digital content as part of the live feed of digitalimages.
 14. The system of claim 10, wherein the operations furtherinclude locating the augmented reality digital content that correspondsto a missing clothing item based on a style of the outfit of the person.15. The system of claim 10, wherein the operations further includelocating the augmented reality digital content that corresponds to amissing clothing item based on a similar style to the outfit of theperson.
 16. The system of claim 10, wherein the located augmentedreality digital content is selectable to purchase the missing item. 17.One or more computer-readable storage media storing instructions thatare executable by a computing device to perform operations including:training a machine-learning model using a plurality of digital imagesdepicting persons wearing outfits that include at least one clothingitem; receiving an additional digital image subsequent to training themachine-learning model, the additional digital image being included in alive feed of digital images capturing an outfit of a person; determiningthat at least one common clothing item in the digital images of similaroutfits to the outfit is missing from the outfit of the person based onthe additional digital image and the trained machine-learning model;locating augmented reality digital content that corresponds to a missingclothing item of the outfit of the person; and outputting the locatedaugmented reality digital content as part of the live feed of digitalimages.
 18. The one or more computer-readable storage media of claim 17,wherein the outfit and the similar outfits include a similar top. 19.The one or more computer-readable storage media of claim 17, wherein theoutfit and the similar outfits include similar bottoms.
 20. The one ormore computer-readable storage media of claim 17, wherein the operationsfurther include: determining that at least one common fashion accessoryin the digital images of similar outfits to the outfit is missing fromthe outfit of the person based on the additional digital image and thetrained machine-learning model; locating additional augmented realitydigital content that corresponds to a missing fashion accessory for theoutfit of the person; and outputting the additional augmented realitydigital content as part of the live feed of digital images.